Storage and compression methods for animated images

ABSTRACT

The present disclosure is related to a method for creating hardware texture content for display on a display of a computing device. The method includes receiving by a processor the content in an image file format, where the content includes a first frame and a second frame. After receiving the content, the method includes creating by the processor a first hardware file corresponding to the first frame and creating by the processor a second hardware file corresponding to the second frame. After creating the first and second frames, the method further includes determining by the processing element a first delta hardware file corresponding to changes between the first hardware file and the second hardware file. In the method the first hardware file, the second hardware file, and the first delta hardware file include data directly usable by a graphics processing unit.

FIELD

The present invention relates generally to rendering of images on adisplay, and more specifically, to rendering of frame-based animatedcomputer graphic images on a display.

BACKGROUND

Computing devices, such as personal computers and mobile electronicdevices, typically include display screens that are used to providevisual output to a user. Often, there is a desire to display contentincluding multiple frames, such as animated content that demonstratesmovement or other variations between frames of the content. Many times,electronic devices are used to access web pages on the internet or othernetwork. For example, many web pages or web-accessible programs (e.g.,web-based video games, interactive websites, and so on), display contentthat includes two or more frames. Examples of this type of contentinclude data displayed or stored as Graphics Interchange Format (GIF),animated portable graphics network (APNG), moving pictures experts groupphase (MPEG or MPG), and so on.

Texture mapping typically involves mapping a source image onto a surfaceof a three-dimensional object, and thereafter mapping the texturedthree-dimensional object to the two-dimensional graphics display screento display the resulting image. Many image file formats can be used asthe source image to create the detail attributes. Examples of attributesthat can be texture mapped include color, specular reflection, vectorperturbation, specularity, transparency, shadows, surface irregularitiesand grading.

Current content storage and display methods for content and texturemapping may require a substantial amount of memory and upload time asdata is transformed from a hardware data to data that can be easily usedand accessed by a graphics processing unit (GPU). This results incontent being rendered and thus displayed slowly, which can result inimage artifacts such as glitches, image tearing, and the like, as wellas resulting in preventing some systems with hardware limitations fromrendering and displaying the content.

SUMMARY

Examples of the disclosure may include a method for creating hardwaretexture content for display on a display of a computing device. Themethod includes a processor receiving the content in an image fileformat, where the content includes a first frame and a second frame.After receiving the content, the method includes the processor creatinga first hardware file corresponding to the first frame and a secondhardware file corresponding to the second frame. After creating thefirst and second frames, the method further includes the processingelement determining a first delta hardware file corresponding to changesbetween the first hardware file and the second hardware file. In themethod, the first hardware file, the second hardware file, and the firstdelta hardware file include data directly usable by a graphicsprocessing unit.

Another example of the present disclosure includes a computing device.The computing device includes a display, a memory component, and a mainprocessing unit in communication with the display and the memorycomponent. The main processing unit is configured to perform operations,including decompressing a content file, creating a plurality of textureframes from the decompressed content file, storing an initial textureframe in the memory component, and storing a plurality of delta framesin the memory component, wherein the plurality of delta frames compriseschange data indicating changes between the initial texture frame and therespective delta frame.

Yet another example of the present disclosure includes a method fordisplaying frame-based content on a display. The method includes agraphics processing unit receiving a hardware animation content file;uploading a first texture frame comprising a first texture to a memorycomponent of the graphics processing unit; directly by the graphicsprocessing unit the first texture from the first texture frame; thegraphics processing unit applying a first delta frame to the firsttexture frame to determine a first texture delta; uploading the firsttexture delta; and the graphics processing unit rendering an outputimage comprising the first texture and the first texture delta overlaidon the first texture.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a system for rendering and displayingcontent.

FIG. 2 is a simplified block diagram of the system of FIG. 1.

FIG. 3 depicts a plurality of frames for content to be displayed by thesystem 100.

FIG. 4 is a flow chart illustrating a conventional rendering andcompression process for animated content.

FIG. 5 is a block diagram illustrating a swizzling operation.

FIG. 6 is a flow chart illustrating a method for rendering images usinghardware compatible data.

FIG. 7 is a diagram illustrating an example of the method of FIG. 6being completed in parallel operations.

FIG. 8 is a flow chart illustrating a method for rendering the contenthardware data.

FIG. 9 is a flow chart illustrating another example of the method ofFIG. 8 utilizing a buffer structure.

FIG. 10 is a flow chart illustrating another example of the method ofFIG. 8 utilizing a double buffer structure.

SPECIFICATION Overview

In some embodiments herein, a method for rendering animated images usinghardware compatible data is disclosed. The method decodes frames ofsource content into one or more hardware texture frames. Once the framesare in a hardware accessible format, the method analyzes each hardwaretexture frame to generate one or more delta frames. The delta framesinclude data related to changes or deltas between an initial frame and aplurality of subsequent frames. For example, a delta frame may includedata corresponding to a portion of the first frame, indicating the areaof the first frame that changed and the data corresponding to themodification to the pixels of that area. After the delta frames havebeen generated, the method further includes compressing and storing thedelta frames. At runtime, or whenever the content is desired to bedisplayed or rendered, the computing device accesses the first hardwarecompressed texture frame and renders the first frame, as the first frameis being rendered or displayed, the delta frames are overlaid on thefirst frame to render the subsequent frames.

In this embodiment, because the initial frame and the delta frames arestored as hardware compressed textures, the data is directly accessibleby the processing element responsible for rendering the content (e.g.,the GPU). This allows the GPU to directly use the frames, withoutrequiring time-consuming swizzling, or other modifications to the datato transform the data into a hardware-friendly format. Further, thecompressed hardware textures require less storage as compared to theuncompressed content, which reduces the amount of memory needed for thecontent.

The method may be used to format and compress image data for a varietyof frame-based content, such as movies, animations, video clips, videogames, computer-aided animation programs, virtual reality, or the like.These types of content may be accessible via a network or may be storedand rendered on the device itself (e.g., a video game). The methoddescribed herein may allow subsequent frames in a content to be renderedwhile the first frame or previous frame is being used for rendering,which helps to reduce glitches, artifacts in the displayed content, andprevents the images from tearing as well as helps to prevent frame ratestuttering.

As used herein the term “texture” is meant to encompass data, such asimage data, that uses red, green, blue, and optionally alpha (RGB(a))values or other color spaces (e.g., cyan, magenta, yellow, and key(CMKY), HSL, etc.) to assign pixels of a display a particular color. Inshort, the term texture is meant to encompass data for mapping surfacedetail or color (e.g., texture mapping) onto an object, as well as datafor creating an image and objects within the image. In some instancesthe term texture may encompass instances where the color space usedinitially may change to another color space when compressed. A texturemay be substantially anything rendered by Open Graphics Library(OpenGL), Microsoft Direct X, {hacek over (S)}², or other programminginterfaces for rendering two-dimensional and three-dimensional graphics.Additionally, the term “texture compression format” is meant toencompass data compression that is optimized for random access memory orother memory types that are primarily used by the GPU core or otherparallel rendering component to render an image. For example, data intexture format is able to be decompressed by the GPU, rather thanrequiring a central processing unit (CPU) for decompression, whichallows the image or other data to stay in the compressed format up untilthe image is rendered. It should be noted that the term GPU is meant toencompass components that can perform a plurality of parallelcomputations to render images.

For example, in many instances, texture mapping may apply one or morepoint texture elements (“texels”) to each point element or pixel of thedisplayed portion of the object to which the texture is being mapped.Texture mapping hardware provides information detailing the manner inwhich the texels of a texture map correspond to the pixels on thedisplay screen that represent the object. In some instances, each texelin a texture map is defined by S and T coordinates which identify itslocation in the two-dimensional texture map. For each pixel, thecorresponding texel or texels that map to it are accessed from thetexture map, and incorporated into the final R, G, B values (or othercolor space values) generated for the pixel to represent the texturedobject on the display screen.

It should be understood that each pixel in an object primitive may notmap in a one-to-one correspondence with a single texel in the texturemap for every view of the object. For example, the closer the object isto the view port represented on the display screen, the larger theobject will appear. As the object appears larger on the display screen,the representation of the texture becomes more detailed. Thus, when theobject consumes a fairly large portion of the display screen, a largenumber of pixels are used to represent the object on the display screen,and each pixel that represents the object may map in a one-to-onecorrespondence with a single texel in the texture map, or a single texelmay map to multiple pixels. However, when the object takes up arelatively small portion of the display screen, a much smaller number ofpixels is used to represent the object, resulting in the texture beingrepresented with less detail, so that each pixel may map to multipletexels. Textures may also have three-dimensions, e.g., volumetrictextures, and the depth may be selected based on Z. Textures havingmid-levels may also require computation of their hardware images forthose levels, for example through bilinear interpolation. In addition,each pixel may map into multiple texels when a texture is mapped to asmall portion of an object. Resultant texel data is calculated for eachpixel that maps to more than one texel. As it is common for a pixel tomap to multiple texels, resultant texel data for a pixel typicallyrepresents an average of the texels that map to that pixel. Alsoincluded within the methods of the present disclosure are mipmaps thatare predetermined images accompanying a main texture that can assist inrendering speed and reducing aliasing effects. The definition of textureused herein may encompass mipmaps and they may be processed along withor separate from the texture compression discussed herein.

DETAILED DESCRIPTION

Turning to the figures, a system for compressing and rendering images inaccordance with the present disclosure will now be discussed in moredetail. FIG. 1 is a perspective view of a computing system forcompressing and rendering images. FIG. 2 is a simplified block diagramof the system of FIG. 1. With reference to FIG. 1, the system 100 mayinclude a computing device 102, a display 104, and one or moreinput/output devices such as a keyboard 106 and a mouse 108. Each of thecomponents of the system 100 may be in communication with each other andoptionally other devices, through a wired connection, wirelessconnection, or a combination of wired and wireless connections.Additionally, although the system 100 in FIG. 1 is illustrated as adesktop computing system with the display 104 and input/output devicesbeing separated from each other, in other embodiments, the componentsmay be integrated. For example, the computing device 102 may be a mobileelectronic device such as a smart phone, tablet computer, portablegaming system, or the like. In these examples, two or more of thecomponents may be integrated together.

The computing device 102 may be substantially any type of electronicdevice, such as, but not limited to, a computer, a server, a laptopcomputer, a tablet computer, a smart phone, a portable gaming device, aset top box, a video game console, or the like. Depending on the type ofcomputing device 102 used, the display 104 may be integrated therewithor separate (as shown in FIG. 1).

The display 104 provides a visual output for the system 100 and thecomputing device 102. The display 104 may be, for example, a liquidcrystal display, a plasma display, an organic light emitting diodedisplay, a three-dimensional display, or the like. Additionally, thedisplay 104 may include one or more input mechanisms, for example, oneor more capacitive touch sensors, resistive touch sensors, or the like.In these embodiments, the display 104 may function to provide input andoutput functions for the system 100.

With reference to FIG. 2, the computing device 102 may include one ormore processing elements 110, input/output devices 112, memorycomponents 114, a communication/networking interface 116, a powersupply, and a graphics processing unit 120. Each of the components willbe discussed in turn below.

The one or more processing elements 110 may include a central processingunit, as well as one or more supplementary processing units. Theprocessing element 110 may be substantially any type of electronicdevice capable of processing, receiving, and/or transmittinginstructions. For example the processing element 110 may be amicroprocessor or a microcomputer. Additionally, the processing element110 may include one or more processors. For example, select componentsof the computing device 102 may be controlled by a first processingelement and other components of the computing device 102 may becontrolled by a second processing element, where the first and secondprocessing elements may or may not be in communication with one another.The one or more processing elements 110 may be specialized for selecttasks or may be more generalized, e.g., central processing unit.

The input/output devices 112 may include the keyboard 106 and the mouse108, as well as other peripheral devices that can be used to provideinput to the system 100 and/or computer 102. Examples of input/outputdevices 112 that may be used with the system 110 include a camera,stylus, track pad, joystick, remote control, buttons, switches, and thelike. The input/output devices 112 may allow the user to provide inputto the system 100, as well as provide output to a user as well. Thetypes of input/output device 112 may vary based on the system 100, thecomputing device 102, and the desires of the user.

The memory components 114 are used to store electronic data that can beutilized by the computing device 102. The memory components 114 maystore electronic data or content, e.g., image files, video files, audiofiles, document files, and so on, that correspond to variousapplications or programs. The memory 114 may be, for example,non-volatile storage, a magnetic storage medium, an optical storagemedium, a magneto-optical storage medium, a read-only medium,random-access memory, uniform memory access, flash memory, erasableprogrammable memory, flash memory, or a combination of one or more typesof computer data storage. The computing device 102 may include one ormore memory components 114 that may be specialized or generalized.

The communication/network interface 116 provides communication to andfrom the computing device 102. For example, the communication/networkinterface 116 may transmit data to and from other computing devicesthrough a wireless network (WiFi, Bluetooth, cellular network, etc.) ora wired network (Ethernet).

The power supply 118 provides power to various components of thecomputing device 102. The power supply 118 may include one or morerechargeable, disposable, or hardwire sources, e.g., batteries, powercord, or the like. Additionally, the power supply 118 may include one ormore types of connectors or components that provide different types ofpower to the computing device 102.

The graphics processing unit 120 or GPU may be a component specializedto render images for output by the display 104. The GPU 120 may beintegrated with the computing device 102 and/or the display 104, and mayinclude one or more processing elements 124 and memory 122. Thecomponents 122, 124 of the GPU 120 may be specialized for graphicsrendering. For example, the memory 122 may be similar to the memory 114,but in many instances may be random-access memory that is dual ported(able to be read and written at the same time). In this example, thememory 122 of the GPU 120 stores data for a plurality of pixels of thedisplay 104 for rendering images on the display 104, and also providesbuffering functions to store completed images until they are ready to bedisplayed. Because the GPU 120 includes its own memory 122, the GPU 120can access image data without having to go through the main processingunit 110 to access the main memory 114, which can reduce the speed ofimage rendering. Additionally, although a single GPU 120 has beenillustrated, in some embodiments the computing device 102 may includetwo or more GPUs 120 to provide increased rendering power and speed forthe system 100.

Content that may be displayed and/or rendered by the system 100 will nowbe discussed. FIG. 3 depicts a plurality of frames for content to bedisplayed by the system 100. With reference to FIG. 3, the content 130may include a plurality of frames 132 a-132 n. The frames 132 a-132 nare each images that are output by the display 104 by varying the outputcolors of the various pixels of the display 104. The content 130 may besubstantially any frame-based content, such as, but not limited to,videos, video games, and motion capture systems. In instances where thecontent 130 is a movie or animation, the frames 132 a-132 n may bepredetermined, whereas in instances where the content 130 is interactiveor otherwise utilizes an input, the frames 132 a-132 n may bedynamically modified or created based on the input (e.g., a video gamereceiving a user input corresponding to a character's movement orappearance).

In many embodiments, the frames 132 a-132 n may include similar objects,but may demonstrate the movement of one or more elements of the objectsbetween each frame 132 a-132 n. For example, a first frame 132 a mayillustrate a first object 134 having a first element 136 in a firstposition, the second frame 132 b illustrates the first object 134 withthe first element 136 in a second position, the third frame 132 cillustrates the first object 134 with the first element 136 in a thirdposition, and the final frame 132 n illustrates the first object 134with the first element 136 in a forth position. It should be noted thatalthough FIG. 3 illustrates only the movement of a single element 134through the frames 132 a-132 n in other examples, additional elements,objects, or scenes may be moved or otherwise varied through the frames132 a-132 n.

With reference to FIG. 4, a conventional rendering and compressionprocess for the content 130 illustrated in FIG. 3 will now be discussed.With reference to FIG. 4, the method 200 may begin with operation 202and the first frame 132 a is decompressed into an uncompressed texture.During this decompression, the main memory 114 of the computing device102 is typically used and the decompression process uses a largepercentage of the memory 114. In other words, during this operation 202,the memory 114 may be substantially occupied by the decompressionprocess which reduces the availability of the memory 114 use, typicallyrandom-access memory, for other applications which can introduce latencyand lag into the computing device.

Once the first frame 132 a has been decompressed, the method 200 willtypically proceed to operation 204 and the texture for the first frame132 a is uploaded to the GPU 120. During the uploading process, the GPUmemory 122, which in most instances is the random access memory (RAM)for the GPU 120, will be monopolized by the uploading of theuncompressed texture from the main memory 114. The uploading to the GPUmemory 122 is required in order to render the image as a raw image.

After the uncompressed texture is uploaded to the GPU memory 122, themethod 200 typically proceeds to operation 206. In operation 206, theuncompressed texture of the first frame 132 a is then swizzled into ahardware form. FIG. 5 is a block diagram illustrating a swizzlingoperation. With reference to FIG. 5, a tile of image data 220 thatincludes a color block, intensity bits, and a block of texels (texturepixel elements), is provided to a four-vector register 222. A swizzleoperation is actuated to rearrange the data in additional registers 224,226, and 228 by moving the block 220 laterally and diagonally betweeneach register 224, 226, 228. The movement of the title 220 through theregisters 224, 226, 228 is done in a Z-order (also known as aMorton-order) that maps the multidimensional data into one dimensionwhile preserving the locality of the data. The swizzling operation takestime for the GPU 120 driver to process the data and to transform it intoa hardware form before the data is usable by the GPU 120, which candelay the rendering of the first frame 132 a. Additionally, theswizzling operation is done in a non-linear pattern, which makes thedata output more difficult to predict than a linear operation and thusslows down output processing. In instances where the memory is uniformmemory access, the swizzle operation to generate the Morton order may bedone before or after an upload. The swizzling operation also may be donelinearly in place, meaning that the texture is generated and thenmanipulated prior to use, which may also make the data output moredifficult to predict and slow down output processing.

With reference again to FIG. 4, once the data from block 220 has beenswizzled, the method 200 proceeds to operation 208. In operation 208,the texture corresponding to the first frame 132 a is used. However,typically latency is introduced into the system before the texture canbe used. With reference to FIG. 4, after operation 208, the method 200returns to operation 202 and repeats for the next frame. In this manner,the frame is decompressed, the texture is uploaded to the GPU 120 andthe texture is swizzled in the GPU memory 122 prior to the texture beingable to be used by the GPU 120. This method takes a substantial amountof time and uses large components of both the main memory 114 and theGPU memory 122, which can prevent other applications from running, aswell as introduce latency and other effects into the system. Forexample, in instances where the content 130 may be an animated imageused for a website image, such as an APNG, the APNG will require largeamounts of RAM and uploading to the GPU for rendering, which is costlyfrom a time and resource perspective, additionally, not all hardware cansupport streamed dynamic images at frame rates that are desired.

A method for increasing the upload and rendering time for theframe-based content, such as the content 130 will now be discussed. FIG.6 is a flow chart illustrating a method for rendering images usinghardware compatible data. With reference to FIG. 6, the method 300 maybegin with operation 302 and the content 130 is received by thecomputing device 102. The content 130 may be, for example, an animatedsequence, APNG, movie file, flick book, or substantially any other typeof visual content including one or more frames. Typically, the content130 will be a compressed animation or movie sequence and will bereceived from another computing device, e.g., through thecommunication/network interface 116, or may be accessed from a storagelocation, e.g., memory 114, of the computing device 102.

Once the content 302 has been received by the computing device 102, themethod 300 proceeds to operation 304. In operation 304, the processingelement 110 decompresses the first frame 132 a from its stored orreceived format, e.g., Portable Network Graphics (PNG), jointphotographic experts group format (JPEG), bitmap (BMP), and other imageformats. For example, the processing element 110 applies one or moredecompression algorithms to the first frame 132 a of the content 130 toreturn the frame 132 a to an original form or size from its transportedor stored size. It should be noted that although the content 130 isdiscussed as being decompressed frame by frame, in some embodiments aplurality of frames or all of the frames may be decompressed atsubstantially the same time. The type of decompression will typicallydepend on the type of image file format of the frame 132 a.

After the first frame 132 a has been decompressed, the method 300proceeds to operation 306. In operation 306, the main processing element110 creates a first hardware version of the first frame 132 a. Forexample, the main processing element 110 transforms the data for thefirst frame 132 a from into a 1-3D array of data that can be representedin a specified format, e.g., scalar or vector, the format of the datamay be selected based on the memory type of the GPU memory 122, whichwill allow the first hardware frame 132 a to be easily used by the GPU120 at runtime. Examples of hardware texture file formats include,Ericsson Texture Compression (ETC), PowerVR Texture Compression (PVRTC),and S3 Texture Compression (S3TC)(also known as DirectX TextureCompression (DXTC)). The hardware texture file formats may use a fixedcompression ratio, where the compression is independent of the imagecontents, the fixed compression ratio allows the GPU 120 to randomlyaccess the texture data. In a specific example, the hardware textureformat may be lossy, fixed-rate compression format such as PVRTC.

After operation 306, the method 300 may proceed to operation 308. Inoperation 308 the first hardware frame is stored in the main memory 114of the computing device 102. In instances where the frame is the firstframe 132 a of the content 130, the frame may be stored as a iframe,i.e., the initial frame, or may otherwise be stored so as to indicatethat the frame 132 a is the first frame 132 a of the content, which asdiscussed in more detail below, will be the frame form which subsequentframes are based from.

After operation 308, the method 300 may proceed to operation 310. Inoperation 310, the main processing element 110 determines whether thereis another frame to be decompressed. In instances where the content 130includes animation or movement across frames, there will typically be aplurality of frames for a particular content. If in operation 310,another frame is going to be decompressed, the method 300 proceeds tooperation 312.

In operation 312, the second frame 132 b is decompressed. This operation312 may be substantially the same as operation 304 and the processingelement 110 may apply a decompression algorithm to the original imageformat of the second frame 132 b data in order to decompress it. Afteroperation 312, the method 300 may proceed to operation 314. In operation314, the processing element 110 may create a hardware version of thesecond frame 132 b. Again, this operation 314 may be substantiallysimilar to operation 306, and the processing element 110 may transformthe data corresponding to the second frame 132 b from an image formatinto a hardware accessible file format that may be easily used by theGPU 120.

Once the second frame 132 b has been transformed into the secondhardware frame, the method 300 proceeds to operation 316. In operation316 the processing element 110 determines the delta from the previousframe. Specifically, the processing element 110 analyzes the first frame132 a and the second frame 132 b to determine the changes between thetwo, e.g., the pixels or portions that vary from the first frame 132 a.In one embodiment, the processing element 110 may provide a pixel bypixel comparison of the two frames 132 a, 132 b to determine thechanges. For example, the processing element 110 may do a binary comparebetween a first run-length encoded data string corresponding to thefirst frame 132 a and a second run-length encoded data stringcorresponding to the second frame 132 b to determine the bits thatchange between the two. In this example, with brief reference to FIG. 3,the main processing element 110 would indicate that the pixelscorresponding to the location of the first element 136 have changedbetween the first frame 132 a and the second frame 132 b, but theremaining pixels of the other elements in the image may not havechanged. Thus, the first delta frame may include data corresponding tothe regions of the second frame 132 b that are different from the firstframe 132 a. However, in other embodiments, the processing element 110may compare the frames in other manners.

After the processing element 110 has compared the first frame 132 a andthe second frame 132 b, both in hardware texture form, the method 300proceeds to operation 318. In operation 318, the processing element 110stores the first delta, where the first delta corresponds to thedifferences between the first frame 132 a and the second frame 132 b.The first delta is stored in the memory 114 of the computing device 102and is stored in a hardware compatible format, in particular the,hardware texture file format. The first delta may be a smaller file thanthe first frame 132 a stored in operation 308 as the first delta may thedata corresponding to differences between the first frame 132 a and thesecond frame 132 b and thus data that is the same between the two frames132 a, 132 b may be omitted.

In some instances, the comparisons between two frames may be done beforecreating the hardware textures for those frames, creating a sourcedelta. However, by creating the hardware textures and then determiningthe delta, e.g., analyzing the delta in the hardware space, theoperations done between hardware frames can be reversed. For example, insome instances an XOR combination of two frames may be used to createthe delta frame and by doing this operation in the hardware space theoperation can be reversed to recreate the original frames if desired.

After operation 318, the method 300 may return to operation 310. Forexample, in instances where the content includes three or more frames,operation 310 and operations 312, 314, 316, 318 may be repeated untilall of the frames in the content 130 have been analyzed and the deltasfor each frame have been created. As a specific example, in instanceswhere the content includes a third frame 132 c, the method 300 returnsto operation 312 after the first delta has been created. In thisexample, operations 312, 314, 316, 318 are completed for the third frame132 c with a second delta being determined in operation 316 and storedin operation 318. In one embodiment, the second delta is determined bythe processing element 110 analyzing the differences between the secondframe 132 b and the third frame 132 c. In this example, each subsequentframe is analyzed for differences relative to the previous or adjacentframe. In another example, the second delta frame is determined by theprocessing element 110 analyzing the differences between the first frame132 a and the third frame 132 c. In this example, each subsequent frameof the content 130 is analyzed relative to the initial frame.

If all the frames 132 a-132 d have been analyzed and the deltas for thechanges between each of the frames have been stored and there are noadditional frames to be analyzed, the method 300 proceeds to operation320. In operation 320, the processing element 110 compresses the contenthardware data. In particular, the main processing element 110 compressesthe first hardware frame and the one or more delta frames. For example,the main processing element 110 applies a hardware compression algorithmsuch as Zlib or LZMA, to the data to the compress it. In other examples,substantially any type of loss-less compression system can be used ontop of additional run-length encoded delta compression. In theseexamples, duplicate data points, such as empty spaces on the deltaframes can be removed efficiently.

After operation 320, the method 300 may proceed to operation 322. Inoperation 322, the content hardware data is stored in the main memory114. The content hardware data is stored in a file format that can bedirectly used by the GPU 120, e.g., the texture hardware file format.This allows the content hardware data to be used directly by the GPU 120without requiring reformatting, translation, or modification to the dataprior to rendering. That is, the hardware format of the first hardwarecompressed frame and delta hardware frames allow the frames to bedecompressed by the GPU 120 itself, which allows the frames to remain ina compressed format up until it is used to vary the pixels to displaythe content 130. A method for rendering the content hardware datacreated using the method 300 of FIG. 6 will be discussed in more detailbelow with respect to FIG. 8.

As briefly mentioned above, in some embodiments one or more operationsof the method 300 may be completed in parallel. FIG. 7 is a diagramillustrating an example of the method 300 being completed in paralleloperations. As shown in FIG. 7, the frames 132 a-132 d of the content130 may be decompressed and the hardware versions of the frames 132a-132 d may be created in parallel. However, in other instances, theframes 132 a-132 d may be analyzed sequentially or in another order fromone another.

A method for using the content hardware data created using the method300 of FIG. 6 will now be discussed in more detail. FIG. 8 is a flowchart illustrating a method 400 for rendering the content hardware data.With reference to FIG. 8, the method 400 may begin with operation 402and the GPU 120 fetches the first hardware frame from memory 114. Duringthe fetch operation 402, the first hardware texture data is retrievedfrom memory 114 by the GPU 120. As compared to operation 202 in FIG. 4,operation 402 uses substantially less RAM memory and is faster than thedecompressing times of operation 202. This is because the frame hardwaretexture is compressed and takes less time to transmit, as well asrequires less memory space to be stored in as it is transferred from themain memory 114 to the GPU 120.

Once the GPU 120 has fetched the first hardware frame, the method 400proceeds to operation 404. In operation 404 the first hardware frame isuploaded to the GPU memory 122 or GPU device can include using theglCompressedTexSubImage2D or glCompresssedTexImage2D, which sets theimage in the GPU memory 122 to the image from the CPU. For example, theGPU memory 122 receives the data corresponding to the first hardwaretexture frame from the main memory 114.

After operation 404, the method 400 proceeds to operation 406. Inoperation 406, the GPU 120 uses the first hardware texture frame. Forexample, the GPU 120 renders the first hardware texture frame on thedisplay 104 to provide an output to the user. Continuing with thisexample, once uploaded glActiveTexture can be called on the image thatallows reference to the texture during runtime.

As the first hardware frame is being rendered, the method 400 mayproceed to operation 408. In operation 408, the processing element 124determines whether another frame is to be rendered. If another frame isto be rendered, the method 400 proceeds to operation 410. In operation410 the GPU processing element 124 applies the first delta to the firsthardware frame texture. For example, the GPU 120 retrieves the firstdelta from the GPU memory 122. In some instances logic can be determinedon both directions of play between animations. For example, inembodiments where the delta is determined by an XOR operation, the deltacan be determined both for forward running of the content as well asreverse running of the content (e.g., a rewind type operation). Further,the delta allows a LOOP back or restart of the frame and/or animation.In this example, a loop delta may be determined that takes the lastimage as a delta to the first frame and sets the loopback frame to thesecond frame.

After operation 410, the method 400 proceeds to operation 412. Inoperation 412, the GPU 120 uploads a delta texture region correspondingto the first delta to the GPU processing element 124. In this example,the upload time is substantially reduced as compared to the uploadingtime in the method 200 of FIG. 4. This is because method 300, the entiretexture for the frame is uploaded to set the image in the GPU memory tothe delta image. However, in the method of FIG. 8, the first deltahardware texture is uploaded, which includes data for the changedportions of the first hardware frame texture and so has a reduced sizeas compared to a frame including the entire second frame hardwaretexture. Thus, the data uploaded is smaller which reduces the amount oftime for the upload to complete, as well as reduces the memory requiredfor the uploading process.

After operation 412, the method 400 may proceed to operation 414. Inoperation 414, the GPU 120 renders the existing texture of the firsthardware texture frame with the first delta hardware texture overlaid tocreate the second frame 132 b of the content 130. After operation 414,the method 400 returns to operation 408 to determine if anther frame isto be rendered. If in operation 408 another frame is not desired, themethod 400 proceeds to an end state 416. If in operation 408 anotherframe is desired, the method 400 returns to operations 410, 412, 414which repeat for each frame of the content 130, with the correspondingdelta frames being uploaded and rendered, allowing reference to thetexture at runtime. Because the hardware texture data used by the GPU120 and memory 114 in FIG. 8 is in a format that is readily accessed bythe GPU 120, the content 130 can be rendered and displayed faster andwith less memory resources than is required by the method 200 of FIG. 4.This is because the GPU 120 does not have to swizzle or otherwise modifythe content data into a format that is usable and thus can immediatelyuse the hardware content data.

FIG. 9 is a diagram flow chart illustrating an example of the method400. In particular, FIG. 9 illustrates an example of hardware animationusing a single buffer in place playback.

It should be noted that in some embodiments, a single buffer or amultiple buffer system may be used to render the content hardwaretexture. For example, the methods of FIGS. 8 and 9 illustrate a singlebuffer system where the first hardware texture frame and the subsequentdelta hardware texture frames are uploaded to the GPU memory 122sequentially and the GPU 122 renders each texture from the same buffer,e.g., one in one out. However, in other examples, a double buffer orother multiple buffer structure may be used. For example, FIG. 10illustrates the method of FIGS. 8 and 9 being executed in a doublebuffer structure. In this example, after being fetched from the memory114, the first hardware frame texture is uploaded to a first buffer andthen used by the GPU 120 for rendering. While the first frame texture isbeing rendered, the first delta is applied to the first frame textureand then the second frame hardware texture is uploaded to a secondbuffer. Once the first frame has been rendered, the GPU 120 renders thesecond hardware texture from the second buffer. While the second frameis being rendered, a second delta is applied to the second frame and thethird frame texture is uploaded to the first buffer. After the secondframe has been rendered, the GPU 120 uses the third frame from the firstbuffer to render the third frame. In other words, the two buffersalternate in receiving hardware texture frames for rendering, whichallows the content to be rendered faster as each frame can be ready foruse in one of the buffers while the previous frame is being rendered.

In embodiments where a double buffer system is used, the method allowsrendering to a second frame while the first frame is being used forrendering. The double buffer structure helps to prevent the GPU 120 fromstalling when updating a frame, which helps to prevent image tearing andthe like. For example, in typical MPEG video compression, a stream thatplays into the GPU buffer can tear as the data cannot be updated whilebeing rendered. On the contrary, the double buffer structure of FIG. 10allows for the data to be updated while it is being rendered whichprevents the image from tearing and provides a glitch-free playback.

FIGS. 9 and 10 illustrate methods for displaying the content hardwaredata created in FIG. 6. However, it should be noted that other displayand rendering processes may be used and the above examples are meant asillustrative only.

CONCLUSION

The foregoing description has broad application. For example, whileexamples disclosed herein may focus on animated images, it should beappreciated that the concepts disclosed herein may equally apply toclosing other type of image output formats, such as video files (MPEG)and the like. Accordingly, the discussion of any embodiment is meantonly to be exemplary and is not intended to suggest that the scope ofthe disclosure, including the claims, is limited to these examples.

What is claimed is:
 1. A method for creating hardware texture contentcomprising: receiving by a processor the content in an image fileformat, wherein the content includes a first frame and a second frame;creating by the processor a first hardware file corresponding to thefirst frame; creating by the processor a second hardware filecorresponding to the second frame; and determining by the processingelement a first delta hardware file corresponding to changes between thefirst hardware file and the second hardware file; wherein the firsthardware file, the second hardware file, and the first delta hardwarefile include data directly usable by a graphics processing unit.
 2. Themethod of claim 1, further comprising storing in a memory component thefirst hardware file; and storing in the memory component the first deltahardware file.
 3. The method of claim 2, further comprising compressingthe first hardware file and the first delta hardware file into a contenthardware file.
 4. The method of claim 3, further comprising storing inthe memory component the content hardware file, wherein the contenthardware file corresponds to the content.
 5. The method of claim 1,wherein creating the first hardware file comprises transforming thefirst frame of the content from the image file format into a hardwareformat.
 6. The method of claim 1, wherein determining the first deltahardware file comprises comparing a first set of data corresponding tothe first hardware file to a second set of data corresponding to thesecond hardware file.
 7. The method of claim 1, further comprising:creating by the processor a third hardware file corresponding to a thirdframe of the content; and determining by the processor a second deltafile corresponding to changes between the second hardware file and thethird hardware file.
 8. The method of claim 1, wherein the firsthardware file, the second hardware file, and the first delta file are ina texture hardware forma that is a lossy, fixed-rate texture compressionfile format.
 9. A method for displaying frame based content on a displaycomprising: receiving by a graphics processing unit a hardware animationcontent file; uploading a first texture frame comprising a first textureto a memory component of the graphics processing unit; directly by thegraphics processing unit the first texture from the first texture frame;applying by the graphics processing unit a first delta frame to thefirst texture frame to determine a first texture delta; uploading thefirst texture delta; and rendering by the graphics processing unit anoutput image comprising the first texture and the first texture deltaoverlaid on the first texture.
 10. The method of claim 9, wherein thehardware animation content comprises the first texture frame and aplurality of delta texture frames, wherein the delta texture framescomprise data changes between the first texture frame and a responsivedelta texture frame.
 11. The method of claim 9, wherein the display isintegrated into a mobile computing device.
 12. The method of claim 9,wherein uploading the first texture frame comprises uploading the firsttexture frame to a first buffer in communication with the graphicsprocessing unit.
 13. The method of claim 9, wherein uploading the firsttexture delta comprises uploading the first texture delta to a secondbuffer in communication with the graphics processing unit.
 14. Acomputing device comprising: a display; a memory component; and a mainprocessing unit in communication with the display and the memorycomponent, wherein the main processing unit is configured to perform thefollowing operations: decompressing a content file; creating a pluralityof texture frames from the decompressed content file; storing an initialtexture frame in the memory component; and storing a plurality of deltaframes in the memory component, wherein the plurality of delta framescomprise change data indicating changes between the initial textureframe and the respective delta frame.
 15. The computing device of claim15, wherein the main processing unit further performs the followingoperations: compressing the initial texture frame and the plurality ofdelta frames into a content texture file; and storing the contenttexture file in the memory component.
 16. The computing device of claim15 further comprising a graphics processing unit in communication withthe main processing unit, wherein the graphics processing unit isconfigured to perform the following operations: fetching the contenttexture file; and rendering the content texture file to display thecontent on the display.
 17. The computing device of claim 16, whereinrendering the content texture file comprises: rendering the initialtexture frame; and overlaying each delta frame over the initial textureframe.